Multi-Modality Feature Transform: An Interactive Image Segmentation Approach

Introducing suitable features in the scribble-based foreground-background (Fg/Bg) segmentation problem is crucial. In many cases, the object of interest has different regions with different color modalities. The same applies to a non-uniform background. Fg/Bg color modalities can even overlap when the appearance is solely modeled using color spaces like RGB or Lab. In this paper, we purposefully discriminate Fg scribbles from Bg scribbles for a better representation. This is achieved by learning a discriminative embedding space from the user-provided scribbles. The transformation between the original features and the embedded features is calculated. This transformation is used to project unlabeled features onto the same embedding space. The transformed features are then used in a supervised classification manner to solve the Fg/Bg segmentation problem. We further refine the results using a self-learning strategy, by expanding scribbles and recomputing the embedding and transformations. Finally, we evaluate our algorithms and compare their performance against the state-of-the-art methods on the ISEG dataset with clear improvements over competing methods.